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EXECUTIVE  SUMMARY 


insights  into  tomorrow ” 


Part  of  our  College  mission  is  distribution  of  the 
students’  problem  solving  products  to  DoD 
sponsors  and  other  interested  agencies  to 
enhance  insight  into  contemporary,  defense 
related  issues.  While  the  College  has  accepted  this 
product  as  meeting  academic  requirements  for 
graduation,  the  views  and  opinions  expressed  or  .  . 
implied  are  solely  those  of  the  author  and  should  J  J 

not  be  construed  as  carrying  official  sanction.  J  J 


REPORT  NUMBER  87-2175 
AUTHOR(S)  MAJOR  KENNETH  W.  ROY 

TITLE  A  COMPARISON  OF  THE  EFFECTIVENESS  OF  THE  AIR  COMMAND 
AND  STAFF  COLLEGE  SEMINAR  AND  CORRESPONDENCE  PROGRAMS 


I.  PURPOSE:  This  study  will  first  review  analyses  of  the 
1981-1984  associate  programs,  and  will  provide  a  critical 
evaluation  of  data  obtained  in  previous  attitude  surveys 
regarding  the  effectiveness  of  the  two  associate  programs. 
Previous  studies  will  be  reviewed  for  effectiveness  in 
applying  sound  statistical  principles  to  their  findings,  and 
changes  to  interpretation  of  data  resulting  from  these  surveys 
will  be  highlighted.  Also,  failure  and  completion  rates  for 
the  periods  surveyed  will  be  reviewed  and  compared. 
Additionally,  the  test  results  will  be  analyzed  to  determine 
whether  there  is  any  significant  difference  between  the 
seminar  and  correspondence  programs.  Finally,  for  the  seminar 
option  only,  the  additional  activities  will  be  reviewed  to 
determine  the  percentage  of  those  that  support  lesson  samples 
of  behavior. 


II.  PROBLEM:  Because  the  mission  and  goals  of  all  three 
methods  of  completing  ACSC  are  the  same,  they  should  be 
equally  effective.  For  those  students  that  attend  in 
residence,  there  are  many  opportunities  for  the  faculty  to 
observe  the  student  to  critically  analyze  that  student's 
performance.  However,  for  the  85%  of  officers  enrolled  in  the 
associate  programs,  the  task  of  evaluating  effectiveness  of 
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CONTINUED 


each  of  the  two  methods  is  much  harder.  Because  of  this,  the 
question  arises  as  to  whether  there  is  any  difference  in  the 
effectiveness  of  the  seminar  program  in  accompl i sh i ng  the 
mission  and  goals  of  ACSC  when  compared  to  the  correspondence 
program.  The  primary  hypothesis  of  this  study  is  there  is  no 
significant  difference  in  the  effectiveness  of  the 
correspondence  and  seminar  programs  in  accomplishing  the 
mission  and  goals  of  ACSC.  Further,  the  secondary  hypothesis 
is  there  is  no  significant  difference  in  failure  rates, 
completion  rates,  or  test  scores  for  these  two  options. 

III.  DATA:  The  survey  responses  and  objective  data  indicated 
both  the  seminar  and  correspondence  options  were  equally 
effective  in  accomplishing  the  mission  and  goals  of  ACSC. 
Although  there  was  some  small  variance  between  the  survey 
responses,  that  variance  was  not  statistically  significant. 
Analysis  indicated  no  statistical  difference  in  the  failure 
rates  of  the  two  options  at  the  95%  confidence  level.  On  the 
contrary,  analysis  indicated  there  was  a  statistically 
significant  higher  completion  rate  for  the  seminar  option  than 
the  correspondence  option.  Analysis,  however,  did  indicate  no 
statistical  significance  between  test  scores  for  the  two 
options.  Analysis  indicated  the  additional  activities 
contribute  to  the  learning  process  in  the  seminar  option. 
Indications  from  a  very  limited  sample  support  the  contention 
that  these  additional  activities  support  correctly  answering 
the  test  questions. 

IV.  CONCLUSIONS:  Not  only  did  the  responses  from  the  previous 
surveys  support  the  primary  hypothesis,  so  did  an  extensive 
analysis  of  the  mean  failure  rates  and  average  test  scores. 
Analysis  yielded  no  statistical  difference  between  the  two 
options.  The  only  factor  that  was  significantly  different  was 
the  completion  rates.  Therefore,  although  the  secondary 
hypothesis  is  rejected,  the  primary  hypothesis  of  this  study 
is  accepted. 


CONTINUED 


V.  RECOMMENDATIONS: 

First,  there  should  be  no  major  change  to  either  program. 

Second,  efforts  should  be  taken  by  ECI  and  ACSC  to  raise 
the  completion  rate  for  the  correspondence  program. 

Third,  efforts  should  be  taken  by  ACSC  to  ensure  seminar 
students  profit  from  the  larger  educational  experience 
inherent  in  that  program. 

Fourth,  in  the  seminar  program  the  discussion  questions, 
briefings,  and  videotapes  should  all  be  focused  on  the  samples 
of  behavior  (SOBs)  designed  in  the  lessons. 

Fifth,  the  ACSC  associate  programs  staff  should  continue 
having  resident  students  conduct  surveys  to  determine  opinions 
on  the  effectiveness  of  the  programs. 

Sixth,  future  researchers  should  incorporate  analysis  of 
failure  rates  and  average  test  scores  in  conjunction  with 
their  surveys. 

Seventh,  future  surveys  should  use  a  one-tail  test  for 
the  analysis  of  curriculum  subareas. 

Eighth,  there  needs  to  be  consistency  in  the  design  of 
the  student  populations  to  be  surveyed. 

Finally,  course  managers  for  the  seminar  program  should 
review  their  test  construction  procedures  to  ensure  all  forms 
of  a  test  are  as  equally  constructed  as  possible. 


Chapter  One 


THE  PROBLEM 


BACKGROUND 


Today,  there  are  three  ways  for  an  Air  Force  officer  to 
complete  Air  Command  and  Staff  College  (ACSC).  The  first  way, 
the  most  limited,  is  in  residence.  Under  this  method,  officers 
are  assigned  to  the  Air  University  at  Maxwell  AFB,  Montgomery, 
Alabama,  for  nine  months  of  concentrated  study.  Only 
approximately  15%  of  all  eligible  officers  are  given  the 
opportunity  to  complete  ACSC  in  this  manner.  The  remainder  are 
given  the  opportunity  to  complete  their  professional  military 
education  through  the  associate  programs.  In  this  category 
there  are  two  options  available:  seminar  and  correspondence. 

In  each  of  these  three  methods,  the  mission  is  the  same. 
This  mission  is  "to  enhance  the  professional  knowledge,  skills, 
and  perspectives  of  mid-career  officers  for  increased 
leadership  roles  in  command  and  staff  positions"  (9:2). 
Additionally,  the  senior  staff  of  ACSC  has  established  the 
following  vital  goals  to  ensure  this  primary  mission  is 
accompl i shed : 

1.  To  provide  an  environment  for  personal  and 
professional  growth. 

2.  To  provide  a  forum  for  significant 
professional  contribution. 

3.  To  enhance  knowledge  and  understanding  of 
the  Air  Force's  mission  and  capabilities. 

4.  To  enhance  professional  skills  required  to 
command,  lead,  and  manage  aerospace  forces. 

5.  To  enhance  understanding  of  the  dimension 
and  goals  of  the  military  in  a  democratic 
society  within  the  global  setting. 

6.  To  prepare  for  the  critical  task  of 
imparting  acquired  knowledge,  skills,  and 
perspectives  to  others  (9:2). 


Because  the  mission  and  goals  of  all  three  methods  of 
completing  ACSC  are  the  same,  they  should  be  equally  effective. 
For  those  students  that  attend  in  residence,  there  are  many 
opportunities  for  the  faculty  to  observe  the  student  to 
critically  analyze  that  student's  performance.  However,  for 
the  85%  of  officers  enrolled  in  the  associate  programs,  the 
task  of  evaluating  effectiveness  of  each  of  the  two  methods  is 
much  harder.  Much  of  the  evaluation  is  based  on  the  results  of 
multiple-choice  tests  that  only  evaluate  the  student's  ability 
to  remember  facts.  For  the  correspondence  program,  this  is 
probably  the  most  appropriate  method  of  evaluation.  However, 
in  the  seminar  program,  students  participate  in  and  are  exposed 
to  many  different  additional  activities  not  available  to 
correspondence  students.  These  include  listening  to  guest 
speakers,  sharing  knowledge  with  other  seminar  members,  and 
having  increased  opportunities  to  improve  leadership  skills 
through  the  dynamics  of  group  participation.  Despite  these 
increased  opportunities,  final  evaluation  is  currently 
determined  by  tests  based  on  the  reading  material,  which  is  the 
same  as  for  correspondence  students.  Because  of  this,  the 
question  arises  as  to  whether  there  is  any  difference  in  the 
effectiveness  of  the  seminar  program  in  accomplishing  the 
mission  and  goals  of  ACSC  when  compared  to  the  correspondence 
program.  This  is  the  problem  facing  the  author  in  this 
anal ys  i s . 


This  study  will  first  review  analyses  of  the  1981-1984 
associate  programs,  and  will  provide  a  critical  evaluation  of 
data  obtained  in  previous  attitude  surveys  regarding  the 
effectiveness  of  the  two  associate  programs.  Previous  studies 
will  be  reviewed  for  effectiveness  in  applying  sound 
statistical  principles  to  their  findings,  and  changes  to 
interpretation  to  data  resulting  from  these  surveys  will  be 
highlighted.  Also,  failure  and  completion  rates  for  the 
periods  surveyed  will  be  reviewed  and  compared.  Additionally, 
the  test  results  will  be  analyzed  to  determine  whether  there  is 
any  significant  difference  between  the  seminar  and 
correspondence  programs.  Finally,  for  the  seminar  option  only, 
the  additional  activities  will  be  reviewed  to  determine  the 
percentage  of  those  that  support  lesson  samples  of  behavior. 


OBJECTIVES  OF, STUDY 


T 


\ 

l\  Analyze  results  of  graduate  surveys  that  were 
administered  to  seminar  and  correspondence  students^  Determine 
trends  in  student  attitudes  of  comparative  effectiveness  of  the 
two  options’. 


2>  Determine  if  there  is  a  significantly  higher  failure 
rate  for  the  seminar  option  when  compared  to  the  correspondence 
opt i on  j 


"  3,  Determine  if  there  is  a  significant  difference  in  the 
completion  rate  for  the  seminar  option  when  compared  to  the 
correspondence  optionj  '■>rA- 

-4)  Compare  the  average  exam  scores  for  students  enrolled 
in  the  seminar  and  correspondence  options  and  determine  whether 
they  are  significantly  different.  v  ...  .  — 


5.  Determine  whether  the  expert  briefings,  videotapes, 
class  discussion  questions,  and  student  briefings  in  the 
seminar  option  contribute  to  the  learning  process.  Determine 
whether  these  additional  activities  support  answering  tes.t 
quest i ons . 


ASSUMPTIONS 


The  first  assumption  is  this  study  will  yield  valuable 
information  for  the  ACSC  staff  responsible  for  the  associate 
programs.  Also,  the  findings  will  be  reviewed  by  the  ACSC 
staff  to  further  refine  the  two  associate  programs  to  ensure 
they  are  meeting  the  mission  and  goals  of  the  school. 

The  second  assumption  is  that  the  studies  accomplished  by 
previous  researchers  were  accomplished  using  statistically 
sound  methods.  Since  the  basis  for  much  of  this  analysis  is 
the  written  reports,  as  the  actual  survey  materials  are 
unavailable,  this  author  is  assuming  the  facts,  and  not 
necessarily  the  findings  and  opinions,  of  the  previous  reports 
are  correctly  presented. 

The  third  assumption  is  that  data  gathered  in  the  previous 
studies  still  represent  the  overall  status  and  effectiveness  of 
the  associate  programs  today. 


The  fourth  assumption  Is  that  the  sample  populations  that 
responded  for  the  previous  surveys  accurately  represented 
attitudes  of  the  entire  population. 

The  fifth  assumption  is  that  the  records  maintained  by 
ACSC  and  ECI  accurately  reflect  the  students  enrolling  in, 
failing,  and  completing  the  various  course  phases. 

The  sixth  assumption  is  that  the  test  questions  contained 
in  all  of  the  exams  were  appropriate  in  so  far  as  testing  to 
the  desired  level  of  knowledge /performance . 


LIMITATIONS 


The  first  limitation  is  that  the  majority  of  this  analysis 
is  based  on  surveys  administered  by  personnel  other  than  the 
author.  Therefore,  findings  are  based  on  questions  written  and 
developed  which  may  induce  some  bias.  However,  since  the 
survey  questions  were  reviewed  by  staff  members  of  ACSC  prior 
to  administration  of  the  surveys,  this  limitation  is  somewhat 
reduced . 

The  second  limitation  is  that  surveys  were  accomplished 
from  1982  through  1985,  and  may  not  reflect  exact  perceptions 
of  today's  students.  However,  this  is  necessary  in  order  to 
build  a  trend  analysis  of  the  overall  effectiveness  of  the 
associate  programs. 

The  third  limitation  is  that  the  findings  of  previous 
analyses  were  based  only  on  responses  received  during  the 
survey  process.  There  is  no  way  to  ascertain  the  attitudes  of 
students  that  were  not  selected  as  part  of  the  test  population 
or  failed  to  respond  to  the  questionnaire. 

The  fourth  limitation  was  that  there  were  some  major 
inconsistencies  in  the  administration  and  analysis  of  the 
varied  reports  in  the  past.  These  will  be  identified  and  a 
recommended  procedure  to  correct  these  inconsistencies  and 
present  a  unified  analysis  for  this  and  future  investigations 
of  this  topic  will  be  included  in  the  following  chapters. 

The  fifth  limitation  is  that  the  success  of  each  of  these 
two  options  is  judged  only  in  light  of  test  scores  on 
end-o f-course  exams.  There  is  no  consideration  given  to  the 
experiences  gained  by  seminar  students  in  the  leading  of 
specific  seminars,  presenting  briefings,  hearing  expert/guest 
speakers,  and  participating  in  the  larger  seminar  experience. 


HYPOTHESES  OF  THIS  STUDY 
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The  major  hypothesis  of  this  study  is  there  is  no 
significant  difference  in  the  effectiveness  of  the 
correspondence  and  seminar  programs  in  accomplishing  the 
mission  and  goals  of  ACSC.  Further,  the  minor  hypothesis  is 
there  is  no  significant  difference  in  failure  rates,  completion 
rates,  or  test  scores  for  these  two  options. 


CONTENTS  OF  THIS  STUDY 


Chapter  I  outlines  the  basic  thesis  of  the  study  as  well 
as  presents  the  background,  questions  to  be  studied,  and 
assumptions  and  limitations. 

Chapter  II  is  a  review  of  previous  analyses.  Also 
included  is  a  brief  discussion  of  basic  statistical  measures 
appropriate  for  this  study. 

Chapter  III  details  the  methodology  used  in  analyzing  the 
data  pertaining  to  this  study.  This  Includes  not  only  data 
contained  in  previous  studies,  but  also  introduces  additional 
measures  such  as  failure  rates,  completion  rates,  and  average 
test  scores  Into  this  analysis.  Also  included  is  an  analysis 
of  additional  activities  distinct  to  the  seminar  program. 

Chapter  IV  is  a  discussion  of  trends  discovered  during 
this  study.  This  will  relate  to  findings  from  previous  studies 
as  well  as  findings  ascertained  from  analysis  of  available 
data . 

Chapter  V  contains  conclusions  and  recommendations  for 
ACSC  Associate  Program  managers. 
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REVIEW  OF  RELATED  LITERATURE 


Several  studies  on  this  topic  have  been  conducted  in 
recent  years.  However,  only  those  studies  accomplished  since 
1982  will  be  included  in  this  report.  These  studies  are 
briefly  recapped  in  this  chapter. 
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SEMI 


In  1982  Majors  David  W.  Alexander  and  Crandler  Crowson, 

Jr,  conducted  a  survey  of  both  the  seminar  and  correspondence 
programs.  In  this  survey,  they  based  their  analyses  on  the 
return  of  552  questionnaires  from  the  seminar  students,  and  692 
returned  questionnaires  from  correspondence  students.  For  the 
seminar  program,  the  target  population  was  only  those  students 
that  had  completed  the  seminar  program  in  AY81.  In  the 
correspondence  program,  questionnaires  were  administered  to 
those  students  that  were  enrolled  in  either  Phase  B,  C,  or  D 
<  1  :8)  . 

This  study  made  the  following  recommendations  for  the 
seminar  program: 

1.  Decrease  the  Staff  Communications  phase  of  the 
program,  and  reduce  the  elementary  aspects  such  as  emphasis  on 
grammar  and  composition. 

2.  Review  the  Quantitative  Methods  Instruction  segment 
for  applicability  to  the  school  mission. 

3.  Update  the  videotapes  used  in  the  course,  and  delete 
the  "Air  Power  Heritage*  videos. 


4.  Reduce  emphasis  on  management-related  topics,  with  a 
corresponding  Increase  on  command  and  leadership  topics. 

5.  Provide  additional  training  to  faculty  instructors 
(FIs)  so  they  may  be  better  prepared  for  their  responsibilities 
to  the  seminars.  Review  the  perceptions  seminars  have  for 
their  associated  FIs. 

6.  Decrease  course  length  to  six  months,  primarily  by 
conducting  two  seminars  a  week. 

7.  Reduce  the  reading  materials  by  editing  those  that  are 
judged  to  be  excessively  long  and  tedious. 

8.  Finally,  develop  a  method  to  keep  the  readings  as 
current  and  applicable  as  possible  (1:33-34). 


Correspondence  Program 


For  the  correspondence  program,  the  ratings  were  somewhat 
lower  than  for  the  seminar  program.  There  appeared  to  be  less 
satisfaction  with  each  phase  of  instruction.  Specifically,  the 
correspondence  course  students  judged  the  reading  material  to 
be  lower  in  quality  than  seminar  students. 

This  study  made  the  following  recommendations  for  the 
correspondence  program: 

1.  Decrease  the  Staff  Commun i cat i ons  phase  of  the 
program,  and  reduce  the  elementary  aspects  such  as  emphasis  on 
grammar  and  composition.  Eliminate  the  requirement  for  the 
students  to  deliver  a  military  briefing. 

2.  Review  the  Quantitative  Methods  instruction  segment 
for  applicability  to  the  school  mission. 

3.  Reduce  the  reading  materials  by  editing  those  that  are 
judged  to  be  excessively  long  and  tedious. 

4.  Increase  the  coverage  of  studies  on  the  USSR  and  the 
PRC,  International  Law,  Tactical  Air  Forces,  and  Strategic  Ait 
Forces  . 

5.  Review  exams  to  change  the  emphasis  from  “laundry 
list"  type  questions  to  conce p t - te s t i ng  questions. 

6.  Develop  a  method  for  the  students  to  trigger  the 
forwarding  of  their  end-o f-course  test,  without  having  to 
submit  the  end-of-phase  critique. 
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7.  Finally,  develop  a  method  to  keep  the  readings  as 
current  and  applicable  as  possible  (1:36). 


Summary 


Generally,  the  findings  for  each  method  were 
no  major  differences  determined  between  the  two. 
judged  the  overall  quality  of  the  programs  to  be 
over  55%  of  the  respondents  giving  an  assessment 
'excellent*  or  “outstanding".  The  most  positive 
seminar  program  was  the  opportunity  for  the  students  to 
interact  with  officers  with  other  career  backgrounds  and 
experiences  (1:22). 


positive,  with 
The  surveys 
good,  with 
rating  of 
feature  of  the 


1983  STUDIES  OF  SEMINAR  AND  CORRESPONDENCE  PROGRAMS 


Seminar  Program 


In  1983,  and  later  years,  separate  studies  were  made  of 
the  correspondence  and  seminar  programs.  The  study  of  the 
seminar  program,  by  Major  Michael  P.  McCall,  stated  that  the 
seminar  program  failed  to  meet  its  main  objectives.  The 
findings  Indicated  to  McCall  that  only  the  Staff  Communications 
and  Command,  Leadership,  and  Resource  Management  phases  of  the 
curriculum  were  effective  in  supporting  the  school  objectives. 
He  further  stated  the  hypotheses  that  the  National  Security 
Affairs,  Warfare  Studies,  and  overall  Seminar  Performance  were 
effective  in  supporting  the  mission  were  rejected.  The  reason 
for  these  findings  were  based  on  Improper  selection  of 
statistical  measures  of  Interpretation  that  will  be  corrected 
in  chapter  five  of  this  study. 

The  primary  recommendations  McCall  made  in  his  study  were: 

1.  Improve  and  standardize  faculty  critiques  of  writing 
ass ignments  . 

2.  Strengthen  the  briefing  program  to  ensure  more  success 
of  the  seminars. 

3.  Rewrite  the  statistics  portion  of  the  curriculum,  or 
delete  It. 

4.  Replace  'boring'  videotapes. 
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5.  Edit  reading  materials  to  ensure  they  only  contain 
materials  that  directly  support  the  learning  objectives.  Also, 
include  audiotapes  on  a  permanent  basis  to  the  seminars  to 
complement  the  readings. 

6.  Test  the  students  four  times  instead  of  two,  and 
design  the  exams  to  test  for  comprehension  and  knowledge  rather 
than  the  ability  to  remember  details. 

7.  Strengthen  the  unannounced  visitation  program  (7:25). 


Correspondence  Program 


The  study  on  the  correspondence  program,  conducted  by 
Major  Robert  L.  Joyal,  found  the  program  to  be  meeting  the 
school  objectives.  Joyal  surveyed  only  those  officers  that  had 
completed  all  four  phases  of  the  program  in  FY82.  In  his 
analysis,  he  determined  that  only  the  Staff  Communications 
phase  was  not  adequately  supporting  the  school  objectives,  but 
overall,  the  correspondence  course  was  accomplishing  its 
mission.  He  then  made  the  following  recommendations: 

1.  Review  the  Staff  Communications  curriculum  and 
consider  eliminating  the  requirement  for  a  student  briefing. 

2.  Review  and  update  the  Quantitative  Methods  lesson. 

3.  The  Leadership  and  Management  portion  of  Instruction 
should  be  expanded. 

4.  Revise  exams  to  focus  on  concept  application  rather 
than  memorization  of  "laundry  lists."  Also,  raise  the  passing 
grade  from  65%  to  70%. 

5.  Raise  the  postal  priority  of  course  materials  for 
overseas  students. 

6.  Review  the  quality  of  the  reading  material  to  ensure 
they  support  the  objectives  of  the  school  (6:20-21). 


Summary 


According  to  the  1983  student  surveys,  both  the  seminar 
and  correspondence  programs  were  fairly  equally  accomplishing 
the  mission  of  ACSC.  The  recommendations  for  the  seminar 
program  were  directed  more  to  program  administration  than  the 
course  content.  However,  surveys  for  both  the  seminar  and 


In  Major  John  T.  Huguley,  Jr.'s,  1984  study  of  the  ACSC 
seminar  program,  he  found  this  option  was  effective,  with  over 
59%  of  survey  responses  judging  the  program  to  be  either 
"outstanding*  or  "excellent".  In  this  study,  the  Staff 
Communications  block  was  judged  to  be  the  least  effective  block 
of  instruction.  The  Command,  Leadership,  and  Resource 
Management  block  received  more  favorable  responses,  and  the 
National  Security  Affairs  and  Warfare  Studies  blocks  received 
the  highest  ratings.  The  primary  recommendations  made  in  this 
study  were: 

1.  Once  again,  improve  writing  critiques  of  student 
papers . 

2.  Review  guidance  on  procedures  for  conducting  effective 
br ie  f  ings  . 

3.  Simplify  instruction  on  the  systems  acquisition 
process  . 

4.  Reduce  quantity  of  course  materials. 

5.  Eliminate  unannounced  visit  program  [note:  this  is 
opposite  to  the  recommendation  of  the  1983  study}. 

6.  Update  videotapes. 

7.  Insure  administration  computer  products  are  timely, 
accurate,  and  complete. 

8.  Improve  quality  control  of  exams  (4:29-30). 


Correspondence  Program 

The  study  of  the  correspondence  program  by  Major  David  R. 
Casteel  found  this  option  to  be  extremely  positive.  Major 
Casteel  determined  the  program  was  effective  in  meeting  the 
mission,  goals,  and  objectives  of  ACSC.  In  this  study  the 


National  Security  Affairs  block  was  the  most  positively  rated 
area#  with  Warfare  Studies  close  behind.  Once  again#  the  Staff 
Communications  block  was  rated  the  lowest.  In  general,  over 
56%  of  the  respondents  believed  the  course  had  helped  In 
Increasing  their  officership  qualities.  In  this  study  the 
primary  recommendations  were: 

1.  Ensure  course  material  is  sent  to  students  in  a  timely 
manner . 

2.  Allow  local  base  education  officers  to  grade  exams 
on-the-spot . 

3.  Increase  emphasis  on  effectively  writing  an  OER. 

4.  Mail  diplomas  directly  to  the  student's  commander. 

5.  Continue  to  stress  the  attributes  of  professionalism. 

6.  Monitor  attitudes  of  supervisors  to  ensure  support  of 
the  correspondence  program. 

7.  Eliminate  “trusted  critic"  evaluation  of  student 
brief ings. 

8.  Consider  raising  the  passing  grade  for  the  exams  to 
75%  or  80%  (2:30-31). 


Summary 


In  summary.  In  1984  both  programs  were  judged  to  be 
excellently  meeting  the  mission  of  ACSC.  The  majority  of 
survey  respondents  felt  the  programs  were  either  "outstanding" 
or  "excellent".  In  reviewing  the  recommendations#  It  appears 
they  were  still  rather  oriented  to  administration  of  the 
courses  rather  than  focusing  on  course  content. 


1985  STUDIES  OF  SEMINAR  AND  CORRESPONDENCE  PROGRAMS 


Seminar  Program 


Major  David  F.  Hulme  surveyed  students  enrolled  In  the 
1984  seminar  program  and  determined  the  program  met  the  mission 
of  ACSC.  In  this  analysis  the  most  favorably  received  aspects 
were  the  faculty  instructor  participation  and  the  inclusion  of 


videotapes  In  the  curriculum.  The  weakest  areas  were  the 
instructional  lessons  on  briefings  and  time/stress  management, 
the  visitation  program,  and  the  formal  testing  program.  This 
study  made  the  following  recommendations: 

1.  The  lessons  on  Staff  Briefing  Skills  and  Stress/Time 
management  be  revalidated. 

2.  ACSC/EDV  totally  review  the  formal  examination 
program. 

3.  The  visitation  program  be  changed  to  provide  for  one 
unannounced  and  two  announced  visits  per  year  (5:41). 


Correspondence  Program 


The  study  of  the  correspondence  program,  conducted  by 
Major  John  L.  Grumbles,  determined  the  program  to  be  rated  even 
higher  in  1984  than  in  1983.  Once  again,  the  National  Security 
Affairs  and  Warfare  Studies  blocks  received  the  highest 
ratings.  All  the  other  areas  also  received  positive  ratings, 
thus  showing  that  the  course  program  managers  were  implementing 
recommendations  from  previous  surveys.  The  following 
recommendations  were  made  in  this  study: 

1.  Allow  on-the-spot  grading  by  the  local  base  education 
office. 

2.  Provide  more  course  curriculum  on  command,  leadership, 
and  offlcership. 

3.  Continue  good  quality  control  of  reading  material. 

4.  Analyze  phase  point  at  which  the  course  is  most 
valuable  to  the  students. 

5.  Standardize  parts  of  surveys  administered  to  resident, 
correspondence,  and  seminar  students  to  permit  comparison  of 
results  and  trend  analysis  (3:35-36). 


Summary 


Thus,  in  summary,  once  again  both  programs  were  rated  very 
highly.  The  trend,  from  the  students'  points  of  view  at  least, 
is  that  the  seminar  and  correspondence  programs  were  Improving. 
This  showed  a  responsiveness  of  the  program  managers  to  listen 
to  input  from  students  and  make  the  necessary  changes  to  more 
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effectively  accomplish  the  ACSC  mission. 


TRENDS 


From  this  review  it  can  be  seen  that  generally  the 
students  had  a  favorable  opinion  of  the  seminar  and 
correspondence  programs.  There  were  several  recommendations 
that  were  adopted  by  program  managers  over  the  years,  and  the 
results  of  these  showed  up  in  the  subsequent  surveys.  There 
were  also  many  adm i n i s trat i ve -or iented  recommendations  that  did 
not  stand  the  test  of  practicality,  such  as  conducting  two 
seminars  a  week,  which  were  not  adopted.  Although  both  seminar 
and  correspondence  options  were  determined  to  be  meeting  the 
mission  objectives,  now  additional  data  will  be  reviewed  to 
determine  whether  one  method  is  more  effective  than  the  other. 


APPROPRIATE  STATISTICAL  MEASURES 


In  previous  analyses,  the  researchers  employed  a  two-tail 
test  to  determine  whether  the  associate  programs  were  meeting 
the  objectives  of  ACSC.  In  this,  the  hypothesis  was  accepted 
if  the  responses  fell  at  or  above  the  95%  confidence  level  of 
the  normal  distribution.  However,  in  one  case  at  least,  this 
led  to  the  erroneous  rejection  of  a  survey  hypothesis. 
Therefore,  a  recommended  approach  will  be  presented  in  chapter 
five.  In  the  analysis  of  failure  rates,  completion  rates,  and 
comparison  of  test  scores  for  the  two  options,  a  two-tail  test 
will  be  employed.  Below  are  several  statistical  measures,  with 
explanations  developed  by  the  author  to  present  them  in 
layman's  terms,  that  will  be  used  in  the  analysis  of  the 
available  data. 

SURVEY  SAMPLE  SIZE  -  Throughout  the  previous  surveys,  one 
of  the  primary  considerations  was  that  the  number  of  course 
officers  responding  to  the  quest i onn i res  would  be  statistically 
significant.  This  would  be  true  if  a  confidence  level  of  95% 
was  achieved,  with  a  plus  or  minus  5%  precision  level.  In  each 
of  the  surveys  reviewed,  this  number  of  required  responses  was 
exceeded,  therefore  giving  credence  to  the  survey  results. 

MEAN  -  This  is  also  known  as  the  mean  index  value  by  some 
researchers  .  This  is  basically  the  arithmetic  average  of  a 
set  of  numbers  (4:7;  6:8;  7:6). 


STANDARD  DEVIATION  -  This  is  the  square  root  of  the 
average  squared  deviations  from  the  mean.  It  signifies  how 
closely  the  mean  represents  the  various  occurrences  in  a 
distribution  <4:7;  6:8;  7:6). 

NORMAL  DISTRIBUTION  -  It  is  assumed  that  responses  from 
survey  questions,  and  data  regarding  failure  rates,  completion 
rates,  and  test  scores  are  normally  distributed  about  their 
respective  averages. 
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METHODOLOGY  AND  INTRODUCTION  OF  DATA 


The  previous  chapter  presented  some  of  the  subjective 
assessments  made  by  various  researchers.  Now  the  quantitative 
data  developed  from  these  previous  surveys  will  be  considered. 
Add i t i onal 1 y ,  data  regarding  the  failure  rates  for  the  two 
associate  options,  their  corresponding  completion  rates,  and 
the  average  test  scores  received  by  students  during  this  period 
will  be  introduced.  Also,  the  additional  activities  in  the 
seminar  program  will  be  reviewed,  as  well  as  the  potential 
influence  they  have  on  addressing  the  samples  of  behavior  and 
correctly  answering  exam  questions.  The  findings  from  the 
review  of  these  data  will  be  presented  in  chapter  four  of  this 
paper . 


PRESENTATION  OF  SURVEY  DATA 


Throughout  this  period  of  research,  there  were  major 
consistencies,  as  well  as  some  inconsistencies,  in  the  way  the 
student  surveys  were  conducted  and  interpreted.  One  constant 
attribute  was  the  fact  that  each  of  the  survey  results  was 
based  on  a  statistically-determined  sample  population  that  was 
valid  at  the  95%  confidence  level. 

In  both  the  seminar  and  correspondence  options,  the 
courses  were  divided  into  various  sub-areas.  These  were  Staff 
Communications;  Command,  Leadership,  and  Resource  Management; 
National  Security  Affairs;  and  Warfare  Studies.  Each  of  these 
sub-areas  was  individually  reviewed  by  the  previous 
researchers . 

In  the  earlier  years  the  researchers  used  such  statistical 
measures  as  the  mean  index  value  <MIV)  (which  is  basically  the 
mean),  the  standard  error  of  the  mean  (which  is  the  standard 
deviation  divided  by  the  square  root  of  the  sample  size),  and 
confidence  intervals  (which  adds  a  degree  of  confidence  that 
the  sample  MIVs  are  sound  estimates  of  the  population  MIVs.) 


J  These  studies  evaluated  the  responses  from  the  seminar  and 

•  correspondence  course  student  surveys,  and  postulated  whether 

I  each  curriculum  sub-area  was  effectively  meeting  the  mission  of 

ACSC . 

Later  studies  focused  on  the  statistical  measures  of  the 
mean  (the  arithmetic  average  of  responses)  and  the  mode  (the 
I  most  frequent  response  to  a  question)  as  the  primary  response 

I  measurements.  The  findings  were  then  simply  analyzed  to 

;  determine  whether  the  tendencies  of  student  opinions  for  each 

1  subarea  of  curriculum  were  strongly  positive,  positive, 

!  neutral,  negative,  or  strongly  negative. 

!  Throughout  the  survey  years  there  was  little  consistency 

I  in  determining  the  appropriate  student  populations  to  be  the 

!  subjects  of  analysis.  Populations  were  based  on  students 

]  completing  various  phases  in  either  the  previous  calender,  or 

fiscal,  year,  completing  during  the  period  1  Jan  -  31  Oct,  or 
J  enrolled  in  one  of  the  phases  of  the  courses.  Therefore,  while 

•  these  populations  are  close  in  composition  and  differences  are 

j  probably  not  statistically  significant,  this  factor  is  worth 

noting.  In  chapter  five  of  this  paper  a  recommended  standard 
for  this  factor  in  future  surveys  will  be  presented. 

However,  to  determine  trends  in  the  results  of  previous 
studies,  there  must  be  some  consistency.  Therefore,  individual 
j  survey  results  must  be  presented  so  they  may  be  compared  to  one 

'  another.  To  do  this,  all  survey  responses  have  been  recomputed 

for  analysis  here.  The  means  of  the  responses  to  the  opinions 
of  students  of  curriculum  subareas  are  thus  presented  below. 

The  data  were  recomputed  so  that  the  responses  were  based  on  a 
5-point  Likert  scale,  with  "A”  meaning  "strongly  positive"  for 
I  5  points,  "B"  meaning  "positive"  for  4  points,  "C"  meaning 

1  "neutral"  for  3  points,  "D"  meaning  "negative"  for  2  points, 

and  "E"  meaning  "strongly  negative"  for  1  point.  Only  the  data 
are  presented  in  Table  1,  with  trends  and  findings  to  follow  in 
chapter  four. 
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S  "S 


Correspondence 

Se  m i nar 

1981 

Mission 

3.800 

3.915 

Staff  Coram 

3.305 

3.614 

Cmd,  Ldrshp, 

&  Res  Mgmt 

3.529 

3.680 

Nat  Sec  Aff 

3.523 

3.820 

Warfare  Stud 

3.506 

3.788 

1982 

Mission 

3.702 

3.381 

Staff  Comm 

3.264 

3.552 

Cmd,  Ldrshp, 

&  Res  Mgmt 

3.519 

3.548 

Nat  Sec  Aff 

3.678 

3.807 

Warfare  Stud 

3.610 

3.742 

1983 

Mission 

3.480 

3.964 

Staff  Comm 

2.990 

3.573 

Cmd,  Ldrshp, 

&  Res  Mgmt 

3.550 

3.630 

Nat  Sec  Aff 

3.770 

3.630 

Warfare  Stud 

3.780 

3.856 

1984 

Mission 

3.340 

4.001 

Staff  Comm 

3.100 

3.684 

Cmd,  Ldrshp, 

&  Res  Mgmt 

3.600 

3.733 

Nat  Sec  Aff 

3.860 

4.063 

Warfare  Stud 

3.770 

3.996 

Table  l.  Mean  Survey  Responses  of  Curriculum  Sub-areas  for 

Years  1981-1984 


As  can  be  seen  above,  there  was  very  little  variance 
between  the  two  options.  Nearly  all  of  the  sub-areas  were 
given  positive  ratings,  showing  there  was  basically  acceptance 
by  the  student  populations  in  the  format  and  content  of  each 
sub-area . 


PRESENTATION  OF  FAILURE  AND  COMPLETION  RATES 


In  addition  to  survey  results,  there  is  other  information 
which  can  help  determine  whether  the  seminar  and  correspondence 
programs  are  meeting  mission  goals.  As  discussed  above,  for 
both  options  the  curriculum  is  divided  into  four  sub-areas. 
Until  1983,  the  seminar  program  had  only  two  tests  for  the 
material.  In  1983,  the  seminar  option  offered  a  four-phase 
course,  which  is  the  same  as  had  been  offered  in  the 
correspondence  option  throughout  this  entire  period. 

Therefore,  in  subsequent  discussion,  each  enrollment  in  a  phase 
is  considered  as  enrollment  in  a  separate  course.  Below  are 
shown  the  failure  rates  and  completion  rates  for  the  two 
options.  In  Tables  2-4  below,  any  enrollment  in  any  phase  is 
counted.  Likewise,  every  failure  is  also  counted.  For 
example,  if  a  student  disenrolled  from  the  initial  phase  in  the 
correspondence  option,  then  reenrolled,  subsequently  failed 
each  of  the  four  end  of  course  exams  on  the  first  attempt,  and 
then  passed  the  exams  on  the  second  attempt,  the  tables  below 
would  reflect  two  enrollments,  four  failures,  and  four 
comple  t i ons . 


CORRESPONDENCE 

SEMINAR 

198  1 

8730 

36  1  1 

1982 

12823 

6579 

1983 

18907 

15938 

1984 

17422 

13147 

Table  2.  Students  Enrolling  In  Associate  Programs  for  Years 

1981-  1984  C  9 : -- ) 


70 


1981 

1982 

1983 

1984 


CORRESPONDENCE 
NUMBER/PER  CENT 

109  (1.2%) 
111  (1.8%) 
583  (3.1%) 
1870  < 10.7%) 


SEMINAR 

NUMBER/PER  CENT 

20  ( . 6%) 

14  C  .  2%) 
198  (1.2%) 

2  114  (16.  1  % ) 


Mean  Failure  Rate  4.2% 

Standard  Deviation  3.8 


4 . 5% 
6 . 7 


Table  3.  Students  Falling  a  Phase  in  the  Associate  Programs 

for  Years  1981-1984  (9:--) 


CORRESPONDENCE 

SEMINAR 

NUMBER/PER  CENT 

NUMBER/PER  CENT 

198  1 

4190  (48%) 

3114  (86%) 

1982 

8874  (69%) 

6565  (98%) 

1983 

10353  (55%) 

12448  (78%) 

1984 

11575  (66%) 

12039  (92%) 

Mean  Completion 

Rate  59.5% 

88 . 5% 

}  Standard  Deviation  8.4 

7 . 4 

Table  4.  Students  Completing  a  Phase  In  the  Associate  Programs 

for  Years  1981-1984  (9:--) 


The  tables  above  show  the  relative  size  of  the  student 
populations  in  each  of  the  two  options,  as  well  as  the  relative 
failure  rates  and  completion  rates  during  the  period  of  study. 
As  can  be  seen,  the  correspondence  program  was  the  larger 
program  throughout  the  period.  Also,  while  there  was  not  much 
variance  in  the  average  failure  rates,  there  was  a  much  larger 
difference  in  the  completion  rates  for  this  period. 


PRESENTATION  OF  AVERAGE  TE 


In  reviewing  the  average  test  scores  for  the  two  options, 
several  small  problems  arose.  First,  there  were  no  data 
available  for  the  1981  correspondence  course.  Therefore,  Table 


'f. 


5  Incorporates  1985  data  in  order  to  be  consistent  with  the 
four-year  trends  established  in  other  analyses  in  this  paper. 
Also,  during  this  period  the  seminar  program  was  undergoing  a 
transformation  from  a  two-phase  program  to  a  four-phase 
program . 


CORRESPONDENCE  (FOUR-PHASE)  AVG  TEST 
SCORE/STANDARD  DEVIATION  BY  COURSE  (PHASE) 


1982 

1983 

1984 

1985 


79.462/8.8 
78 . 428/7 . 8 
77 . 808/7 . 1 
78 . 274/7 . 7 


83.181/9 
81.291/8.9 
79.419/8.2 
80.  112/8.7 


84 . 293/8 . 0 
78.415/9.5 
75 . 040/8 . 7 
80.  129/9.  1 


84.591/8.5 
82 . 477/8 . 6 
79 . 750/8 .  1 
80.097/7. 9 


SEMINAR  (TWO-PHASE)  AVG  TEST 
SCORE/STANDARD  DEVIATION  BY  COURSE  (PHASE) 


1982 

1983 

1984 

1985 


78.031/8.0 
69.667/7  .  2 

N/A* 

76 . 000/0** 


82.562/7 . 9 
77.012/10.8 
76.045/9.7 
70. 000/0** 


*  No  students  tested  in  1984 
**  Only  one  student  tested  in  1985 

SEMINAR  (FOUR-PHASE)  AVG  TEST 
SCORE/STANDARD  DEVIATION  BY  COURSE  (PHASE) 


3  1  C 

3  1  D 

3  1  E 

3  1  F 

1982 

N/A 

N/A 

N/A 

N/A 

1983 

77.712/7.7 

77. 901 / 9 . 0 

73.931/7.6 

76.417/7.7 

1984 

77.542/7.6 

78 . 050/9 . 1 

74  .  193/7.6 

76 . 558  / 7 . 3 

1985 

75.148/8.5 

73.754/ 10.0 

80.504/7. 8 

78 . 135/7.9 

Table  5.  Average  Test  Score  Data  for  Years  1982-1985  (10:--) 


In  the  table  above  the  average  test  score  data  are  shewn 
for  the  correspondence,  and  both  the  seminar  two-phase  and 
four-phase,  programs.  As  can  be  seen,  there  was  generally 
little  variance  between  the  programs.  Also,  there  was  a  rathe 
large  standard  deviation  in  each  of  the  years  and  phases,  thus 


signifying  a  relatively  flat  normal  distribution. 


IMPACT  OF  ADDITIONAL  ACTIVITIES  IN  THE  SEMINAR  PROGRAM 


The  final  factor  to  be  considered  In  this  analysis  Is  the 
impact  of  the  additional  activities  in  the  current  seminar 
program.  The  value  the  discussion  questions,  videotapes,  and 
student /expert  briefings  have  In  aiding  in  addressing  the 
samples  of  behavior,  and  subsequently  in  answering  the  end  of 
course  exams,  will  now  be  considered. 

In  determining  these  data,  the  author  first  sampled  the 
lesson  leader  guides  for  all  of  the  odd-numbered  lessons  In  the 
1987  seminar  program  to  determine  the  number  of  discussion 
questions,  videos,  and  briefings  that  were  designed  to  address 
specific  samples  of  behavior  (SOBs).  In  Table  6  below  are 
shown  the  percentages  of  SOBs  that  are  supported  by  either  one 
or  more  of  the  additional  activities  mentioned  above,  as  well 
as  the  actual  test  questions  that  are  likewise  supported.  As 
an  additional  factor,  in  Tables  7  and  8  the  test  question  data 
are  broken  down  into  forms  3  and  4  of  the  current  tests,  and 
the  latest  results  of  the  201  student  samples  are  included  in 
Table  9  for  reference. 


Number 
of  SOBs 


Number  and  % 
Supported 
by  Discussion 
Cues  t  i  ons 


122  (69%) 


Number  and  % 
Supported 
by  Videos 
or  Br i e  f i ngs 


87  (49%) 


Number  and  % 
Supported 
by  Discussion 
and  Videos 
and /or  Briefings 

82  (46%) 


Table  6.  Samples  of  Behavior  Supported  by  Additional 
Activities  In  1987  Seminar  Program  (11:--) 


Pe  rcen  t  age 

Pe  rce  n  t  age 

Pe  rce  n  t  age 

Percent  age 

of  Questions 

Suppor  ted 

Suppor  ted 

Suppor  ted 

with  no 

by  Discussion 

by  Videos 

by  Discussion 

Activities 

Quest  ions 

or  Br i e  f  t  ngs 

and  Videos 

On  1  y 

On  1  y 

and  /  or  Briefings 

1  2% 

36% 

l  8% 

34% 

Table  7.  Analysis  of  Test  Questions  (Form  3 / 1 9 8 7 A Y )  (11 


Percental 
of  Questions 
with  no 
Activities 


Percentage 
supported 
by  Discussion 
Quest  ions 
On  1  y 


Percentage 
supported 
by  Videos 
or  Briefings 
On  1  y 


Percentage 
suppor  ted 
by  Discussion 
and  Videos 
and/or  Briefing 


Table  8.  Analysis  of  Test  Questions  (Form  4/l987AY)  (1 


Form 

Form 

03 

04 

Mean 

77 . 03 

75.18 

Std  Dev 

7 . 85 

9 .72 

Number  Fails 

1  2 

1  5 

1st  Time  Falls 

6  .  OH 

7 . 5  % 

Table  9.  Test  Results  from  First  201  Students  to 
the  36A  Exams  (Forms  03  and  04/1987AY)  (8:--) 


These  tables  show  the  extent  of  the  impact  of  additirn 
activities  in  the  seminar  program  on  a  very  small  sample 
population.  As  stated  earlier,  this  analysis  is  based  on  j 
sampling  from  a  single  year,  and  includes  data  from  jus*  *  h 
initial  201  students  to  take  the  current  tests. 


Now  that  the  raw  data  have  been  presented,  there  u;-.' 
an  analysis  of  what  they  exactly  mean.  The  data  for  the  .  u 
responses,  failure  rates,  completion  rates,  average  test 
scores,  and  impact  of  additional  activities  in  the  seminar 
program  thus  present  a  basis  from  which  several  cone!  us  i  or;, 
be  drawn.  This  will  be  the  topic  of  the  subsequent  chap’er 


Chapter  Four 


FINDINGS  AND  ANALYSIS 


ANALYSIS  OF  SURVEY  DATA 


In  reviewing  the  survey  data  results  presented  In  Table  1 
in  the  previous  chapter,  one  can  discern  several  significant 
Items.  Basically,  the  seminar  students  consistently  rated  that 
program  somewhat  higher  than  did  the  correspondence  students. 
This  Is  true  for  all  20  curriculum  sub-areas  except  for  3, 
those  being  Mission  Effectiveness  In  1982,  and  Staff 
Coaaun icat 1 ons  and  National  Security  Affairs  In  1983. 

In  this  period,  only  one  sub-area.  Staff  Coaaun  i  ca t  i  ons  In 
the  correspondence  prograa  In  1983,  was  judged  to  be  on  the 
negative  side  of  neutral.  All  other  Indicators  show  both  the 
seminar  and  correspondence  students  held  at  least  a  favorable 
opinion  of  the  effectiveness  of  each  curriculum  sub-area. 

Especially  noteworthy  are  the  results  of  the  1984 
responses  on  the  Mission  Effectiveness  and  National  Security 
Affairs  in  the  seminar  prograa.  These  two  areas  were  the  only 
ones  rated  between  positive  and  highly  positive,  and  were  the 
overall  highest  rated  areas  In  the  seminar  prograa  throughout 
this  entire  period.  The  majority  of  the  other  ratings  have 
been  fairly  consistent  throughout  this  survey  period. 

Generally,  these  data  support  the  hypothesis  that  there  is 
no  statistical  significance  between  the  effectiveness  of  the 
two  opttons  in  accomplishing  the  mission  of  ACSC .  There  are 
some  very  minor  variances  between  ratings  for  each  year,  but  in 
no  case  do  they  exceed  .65.  Thus,  based  on  the  review  of  the 
related  literature  in  chapter  two,  and  the  analysis  of  these 
data,  the  findings  support  the  major  hypothesis  of  this  paper. 


ANALYSIS  OF  FAILURE  AND  COMPLETION  RATES 


Fall  ure  Rates 


Reviewing  the  failure  rates.  It  appears  there  Is  a  much 
greater  Incidence  of  failures  In  the  correspondence  program 
than  In  the  seminar  program.  However,  applying  statistical 
analysis,  at  the  95%  confidence  level  there  Is  no  difference 
between  the  two. 

In  analyzing  the  data  from  Table  3,  there  is  a  mean 
failure  rate  for  the  correspondence  program  of  4.2%,  and  a 
standard  deviation  of  3.8.  For  the  seminar  program,  there  Is  a 
mean  failure  rate  of  4.5%,  with  a  standard  deviation  of  6.7. 

To  ascertain  whether  these  two  findings  are  related,  a  95% 
confidence  level  test  was  applied  to  the  these  results  for 
comparison.  In  this  case  the  correspondence  course  results 
were  used  as  the  baseline.  With  a  mean  of  4.2%,  and  a  standard 
deviation  of  3.8,  a  95%  confidence  range  will  Include  all  other 
mean  scores  that  fall  within  1.96  times  the  standard  deviation 
of  the  mean.  This  yields  a  confidence  range  of  4.2  +/-  7.5,  or 
-3.3  to  11.7.  Since  the  mean  for  the  seminar  program  is  4.5%, 
this  is  well  within  the  confidence  level,  and  thus  indicates  no 
statistical  difference  between  failure  rates  for  the  two 
options.  This  indicates  support  for  the  primary  and  secondary 
hypotheses . 


Completion  Rates 


I 

I  In  comparing  mean  completion  rates  for  the  period,  data  in 

Table  4  yields  a  mean  completion  rate  of  59.5%  for  the 
correspondence  program,  with  a  standard  deviation  of  8.4.  For 
the  seminar  program,  there  is  a  mean  of  88.5%,  with  a  standard 
deviation  of  7.4.  Applying  a  95%  confidence  level  test  to 
I  these  indicates  there  is  a  significant  difference  between  the 

two  rates. 

This  is  not  surprising,  however.  The  motivation  for 
completion  in  the  correspondence  course  comes  primarily  from 
within  the  officer.  On  the  other  hand,  in  the  seminar  program 
j  many  students  are  either  "pushed  or  pulled"  Into  completing  the 

course  by  peer  pressure.  Therefore,  this  finding  refutes  the 
secondary  hypothesis. 
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ANALYSIS  OF  AVERAGE  TEST  SCORES 


Review  of  the  average  test  score  data  in  Table  5  reveals 
little  variance  between  the  scores  for  the  correspondence 
course  with  either  the  two  or  four-phase  seminar  program.  A 
visual  analysis  of  the  test  scores  for  the  correspondence 
option  yields  an  approximate  mean  of  80-81\,  with  an 
approximate  standard  deviation  of  8.  Applying  a  95\  confidence 
level  test  to  both  two  and  four-phase  seminar  options  produces 
no  statistical  variance. 

In  order  to  be  significantly  different,  the  scores  would 
have  to  range  below  65,  or  above  95.  Since  none  fall  in  these 
ranges,  we  can  safely  state  there  is  no  significant  difference 
between  test  scores  for  the  correspondence  or  seminar  options, 
and  this  factor  supports  the  primary  and  secondary  hypotheses. 


ANALYSIS  OF  ADDITIONAL  ACTIVITIES  IN  THE  SEMINAR  PROGRAM 


A  major  limitation  in  this  analysis  is  that  the  results 
are  based  on  an  extremely  small  data  base.  The  results  and 
findings  are  based  solely  on  comparing  results  of  the  201 
sample  to  determinations  made  by  the  author  in  identifying 
samples  of  behavior  (SOBs)  that  were  supported  by  additional 
activities. 

In  reviewing  the  data  in  Table  6,  there  is  a  relatively 
high  percentage  of  SOBs  that  were  supported  by  discussion 
questions.  There  was  also  a  good  number  of  SOBs,  nearly  half, 
that  were  supported  by  videos  or  briefings,  or  both  of  these 
plus  discussion  questions. 


Although  the  201  results  of  the  tests  for  forms  03  and  04 
do  not  show  any  statistical  difference,  a  review  of  data  in 
Tables  7  and  8  may  yield  information  as  to  why  there  is  a  lower 
mean,  higher  standard  deviation,  and  higher  first  time  failure 
rate  for  form  04  when  compared  to  form  03.  Review  of  the  data 
shows  the  percentage  of  exam  questions  supported  by  either 
discussion  questions  or  v i deos /br i e f I ngs  is  virtually  the  same. 
However,  in  form  04  the  number  of  test  questions  supported  by 
multiple  activities  is  much  lower,  with  the  number  of  questions 
with  no  support  being  nearly  double  the  percentage  in  form  03. 

Once  again,  although  the  findings  are  based  on  a  very 
small  sample,  they  do  indicate  that  the  additional  activities 
do  impact  on  correctly  answering  the  test  questions.  This, 
however,  needs  to  be  studied  more  extensively  to  make  a 


positive  finding  of  the  net  effect  of  this  on  the  students. 


SUMMARY 


In  this  chapter,  the  basic  findings  and  results  of 
analysis  of  survey  data  from  previous  student  surveys,  the 
failure  and  completion  rates  for  the  two  options,  and  the 
impact  of  additional  activities  in  the  seminar  program  have 
been  presented.  Basically,  there  was  support  for  the  primary 
hypothesis  in  nearly  all  of  the  areas  studied.  Not  only  did 
the  survey  results  indicate  positive  trends  for  the  two 
programs,  so  did  the  more  objective  data  such  as  test  scores 
and  failure  rates.  Chapter  five  will  now  expand  on  this  and 
present  the  final  conclusions  and  recommendations. 
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Chapter  Five 


SUMMARY 


CONCLUSIONS 


la  chapter  one  of  this  paper  two  hypotheses  and  five  major 
questions  were  identified  for  consideration.  The  primary 
hypothesis  of  this  study  is  there  is  no  significant  difference 
in  the  effectiveness  of  the  correspondence  and  seminar  programs 
in  accomplishing  the  mission  and  goals  of  ACSC.  Further,  the 
secondary  hypothesis  is  there  is  no  significant  difference  in 
failure  rates,  completion  rates,  or  test  scores  for  these  two 
opt  ions . 

This  paper  investigated  questions  concerning  the  ACSC 
seminar  and  correspondence  programs.  After  reviewing  all  of 
the  data,  the  findings  are  as  follows: 

Question  1  -  The  survey  responses  and  objective  data 
indicated  both  the  seminar  and  correspondence  options  were 
equally  effective  in  accomplishing  the  mission  and  goals  of 
ACSC.  Although  there  was  some  small  variance  between  the 
survey  responses,  that  variance  was  not  statistically 
significant.  Therefore,  the  primary  hypothesis  is  supported  by 
this  finding. 

Question  2  -  Analysis  Indicated  no  statistical  difference 
in  the  failure  rates  of  the  two  options  at  the  95%  confidence 
level.  Therefore,  the  primary  and  secondary  hypotheses  are 
supported  by  this  finding. 

Question  3  -  Analysis  indicated  there  was  a  statistically 
significant  higher  completion  rate  for  the  seminar  option  than 
the  correspondence  option.  Therefore,  this  finding  does  not 
support,  and  invalidates,  the  secondary  hypothesis. 

Question  4  -  Analysis  indicated  no  statistical 
significance  between  test  scores  for  the  two  options. 


Therefore,  this  finding  supports  the  primary  and  secondary 
hypotheses . 

Question  5  -  Analysis  indicated  the  additional  activities 
contribute  to  the  learning  process  in  the  seminar  option. 
Indications  from  a  very  limited  sample  support  the  contention 
that  these  additional  activities  support  correctly  answering 
the  test  questions.  Therefore,  this  finding  supports  the 
primary  hypothesis. 

Not  only  did  the  responses  from  the  previous  surveys 
support  the  primary  hypothesis,  so  did  an  extensive  analysis  of 
the  mean  failure  rates  and  average  test  scores.  Analysis 
yielded  no  statistical  difference  between  the  two  options.  The 
only  factor  that  was  significantly  different  was  the  completion 
rates.  Therefore,  although  the  secondary  hypothesis  is 
rejected,  the  primary  hypothesis  of  this  study  is  accepted. 


RECOMMENDATIONS 


First,  because  both  the  seminar  and  correspondence  options 
are  judged  to  be  accomplishing  the  goals  and  mission  of  ACSC, 
there  should  be  no  major  change  to  either  program.  Course 
managers  have  actively  sought  to  keep  the  programs  current, 
have  implemented  various  student  suggestions  from  previous 
researchers,  and  should  continue  this  policy. 

Second,  because  there  is  a  much  higher  completion  rate  for 
the  seminar  option  than  the  correspondence  option,  efforts 
should  be  taken  by  ECI  and  ACSC  to  raise  the  completion  rate 
for  the  correspondence  program.  Efforts  such  as  sending  out 
"ticklers-  when  the  student  is  six  and  three  months  away  from 
disenrol lment  may  serve  to  give  the  extra  "push"  required  to 
motivate  some  students  to  completion. 

Third,  although  there  is  no  statistical  difference  between 
the  seminar  and  correspondence  failure  rates  and  test  score 
results,  efforts  should  be  taken  by  ACSC  to  ensure  seminar 
students  profit  from  the  larger  educational  experience  inherent 
in  that  program.  This  could  be  done  by  focusing  test  questions 
for  the  seminar  program  from  the  briefings,  videotapes,  or 
discussion  questions.  Since  the  program  currently  bases  its 
question  solely  from  the  readings,  students  have  no  way  of 
concretely  exhibiting  the  benefits  received  from  the  seminar 
exper ience . 

Fourth,  in  the  seminar  program  the  discussion  questions, 
briefings,  and  videotapes  should  all  be  focused  on  the  samples 


of  behavior  CSOBs)  designed  in  the  lessons.  Although  a  high 
percentage  of  these  currently  support  the  SOBs,  special  effort 
should  be  made  to  ensure  nearly  100%  of  them  are  directly 
related  to  the  SOBs. 

Fifth,  the  ACSC  associate  programs  staff  should  continue 
having  resident  students  conduct  surveys  to  determine  opinions 
on  the  effectiveness  of  the  programs. 

Sixth,  future  researchers  should  incorporate  analysis  of 
failure  rates  and  average  test  scores  in  conjunction  with  their 
surveys.  While  student  opinions  are  good,  these  additional 
factors  would  add  much  to  the  credibility  of  the  analyses  of 
the  two  programs. 

Seventh,  future  surveys  should  use  a  one-tail  test  for  the 
analysis  of  curriculum  sub-areas.  In  one  previous  survey,  the 
hypothesis  that  a  specific  sub-area  supports  the  ACSC  mission 
was  rejected  because  the  responses  fell  above  the  95% 
confidence  level.  This  indicated  there  was  a  higher  student 
assessment  of  that  area  than  the  researcher  had  projected.  Had 
a  one-tall  test  been  employed,  this  error  would  not  have 
occurred . 

Eighth,  there  needs  to  be  consistency  in  the  design  of  the 
student  populations  to  be  surveyed.  Future  researchers  should 
include  all  students  that  have  completed  any  phase  of  either 
course  in  the  previous  fiscal  year.  This  will  coincide  with 
data  tabulated  by  ECI  and  the  ACSC  staff  concerning  student 
enro 1 1 ments . 

Finally,  course  managers  for  the  seminar  program  should 
review  their  test  construction  procedures  to  ensure  all  forms 
of  a  test  are  as  equally  constructed  as  possible.  There  should 
be  an  effort  to  design  the  tests  so  that  the  majority  of  exam 
questions  are  supported  by  the  additional  activities  in  order 
to  capitalize  in  the  time  invested  by  the  students  in  these 
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